Skip to content

[draft] Rename Distribution::HashPartitioned to Distribution::KeyPartitioned#23241

Closed
gabotechs wants to merge 1 commit into
mainfrom
gabotechs/hash-key-rename
Closed

[draft] Rename Distribution::HashPartitioned to Distribution::KeyPartitioned#23241
gabotechs wants to merge 1 commit into
mainfrom
gabotechs/hash-key-rename

Conversation

@gabotechs

@gabotechs gabotechs commented Jun 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Warning

Just trying out what would be the actual impact of the rename, not meant to be merged

Which issue does this PR close?

Rationale for this change

Distribution::HashPartitioned is an unfortunate naming that does not really reflect the real distribution requirement: inputs do not need to be partitioned by a hashing algorithm, they can be partitioned however as long as the same key always falls in the same partition.

What changes are included in this PR?

Just a plain rename Distribution::HashPartitioned -> Distribution::KeyPartitioned

Are these changes tested?

By the compiler mainly, as this is just a rename.

Are there any user-facing changes?

Yes, people consuming Distribution::HashPartitioned need to rename their code to Distribution::KeyPartitioned

Blast radius of the public API change

Findings from GitHub code search: this rename does affect real downstream consumers that construct or pattern-match Distribution::HashPartitioned. Notable examples:

There are additional smaller downstream hits, for example Optiprism, datafusion-bio-functions, and jammi-ai.

@github-actions github-actions Bot added physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate physical-plan Changes to the physical-plan crate labels Jun 29, 2026
@gabotechs gabotechs mentioned this pull request Jun 29, 2026
Closed
@gabotechs gabotechs force-pushed the gabotechs/hash-key-rename branch from cc6c2f5 to c2ab673 Compare June 29, 2026 15:00
@github-actions

github-actions Bot commented Jun 29, 2026 •
edited
Loading

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details 
     Cloning apache/main
    Building datafusion v54.0.0 (current)
       Built [ 104.708s] (current)
     Parsing datafusion v54.0.0 (current)
      Parsed [   0.037s] (current)
    Building datafusion v54.0.0 (baseline)
       Built [ 102.148s] (baseline)
     Parsing datafusion v54.0.0 (baseline)
      Parsed [   0.037s] (baseline)
    Checking datafusion v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.870s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [ 209.274s] datafusion
    Building datafusion-physical-expr v54.0.0 (current)
       Built [  27.265s] (current)
     Parsing datafusion-physical-expr v54.0.0 (current)
      Parsed [   0.051s] (current)
    Building datafusion-physical-expr v54.0.0 (baseline)
       Built [  27.249s] (baseline)
     Parsing datafusion-physical-expr v54.0.0 (baseline)
      Parsed [   0.053s] (baseline)
    Checking datafusion-physical-expr v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.527s] 223 checks: 221 pass, 2 fail, 0 warn, 30 skip

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/enum_variant_added.ron

Failed in:
  variant Distribution:KeyPartitioned in /home/runner/work/datafusion/datafusion/datafusion/physical-expr/src/partitioning.rs:598

--- failure enum_variant_missing: pub enum variant removed or renamed ---

Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/enum_variant_missing.ron

Failed in:
  variant Distribution::HashPartitioned, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/155ab25c6512b68ad06c9ddde2dd95303f1e254d/datafusion/physical-expr/src/partitioning.rs:598

     Summary semver requires new major version: 2 major and 0 minor checks failed
    Finished [  55.971s] datafusion-physical-expr
    Building datafusion-physical-optimizer v54.0.0 (current)
       Built [  36.391s] (current)
     Parsing datafusion-physical-optimizer v54.0.0 (current)
      Parsed [   0.022s] (current)
    Building datafusion-physical-optimizer v54.0.0 (baseline)
       Built [  36.510s] (baseline)
     Parsing datafusion-physical-optimizer v54.0.0 (baseline)
      Parsed [   0.024s] (baseline)
    Checking datafusion-physical-optimizer v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.167s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  74.267s] datafusion-physical-optimizer
    Building datafusion-physical-plan v54.0.0 (current)
       Built [  34.639s] (current)
     Parsing datafusion-physical-plan v54.0.0 (current)
      Parsed [   0.137s] (current)
    Building datafusion-physical-plan v54.0.0 (baseline)
       Built [  34.658s] (baseline)
     Parsing datafusion-physical-plan v54.0.0 (baseline)
      Parsed [   0.140s] (baseline)
    Checking datafusion-physical-plan v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.906s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  71.598s] datafusion-physical-plan

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Jun 29, 2026
@gene-bordegaray gene-bordegaray mentioned this pull request Jun 30, 2026
Merged
@gabotechs

gabotechs commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favor of #23259

@gabotechs gabotechs closed this Jun 30, 2026
lyne7-sc pushed a commit to lyne7-sc/datafusion that referenced this pull request Jul 1, 2026
@gene-bordegaray
Add Distribution::HashPartitioned to Distribution::KeyPartitioned…
15d676a 
… API bridge (apache#23259)

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link to an issue to this PR using the GitHub syntax. For example
`Closes apache#123` indicates that this PR will close issue apache#123.
-->

- Closes apache#23236.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

`HashPartitioned` is historical naming for a key-partitioning
requirement. This keeps the old variant as a deprecated compatibility
bridge while moving DataFusion internals to `KeyPartitioned`.

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

Adds `KeyPartitioned`, deprecates `HashPartitioned`, and treats both
equivalently during the transition to avoid breaking changes for
downstream consumers. A blast radius report on this was done here apache#23241

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

Yes.

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

Yes, `Distribution::HashPartitioned` is deprecated and trainsitioned to
`Distribution::KeyPartitioned`.

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

auto detected api change Auto detected API change core Core DataFusion crate optimizer Optimizer rules physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Replace / rename HashPartitioned distribution as KeyPartitioned

1 participant

@gabotechs